# large language model

Seed-Coder is a series of open-source code large language models launched by the Seed Team of ByteDance. It includes base, instruction, and reasoning models, aiming to significantly enhance programming capabilities through minimal human effort and autonomous management of code training data. The model performs excellently among similar open-source models and is suitable for various coding tasks. It is positioned to promote the development of the open-source LLM ecosystem and is applicable to both research and industry.

ZeroSearch is a novel reinforcement learning framework designed to incentivize the search capabilities of large language models (LLMs) without interacting with actual search engines. Through supervised fine-tuning, ZeroSearch transforms LLMs into retrieval modules capable of generating relevant and irrelevant documents, and introduces a curriculum rollout mechanism to gradually enhance the model's reasoning ability. The main advantage of this technology lies in its superior performance compared to models based on real search engines, while incurring zero API costs. It is suitable for LLMs of all sizes and supports various reinforcement learning algorithms, making it ideal for research and development teams that require efficient retrieval capabilities.

InternVL2-8B-MPO is a multimodal large language model (MLLM) that enhances multimodal inference capabilities by introducing a Mixed Preference Optimization (MPO) process. The model features an automated pipeline for preference data construction and builds the MMPR, a large-scale multimodal inference preference dataset. Based on the InternVL2-8B model, InternVL2-8B-MPO is fine-tuned using the MMPR dataset, demonstrating stronger multimodal inference capabilities with fewer hallucinations. The model achieved an accuracy of 67.0% on MathVista, surpassing the InternVL2-8B by 8.7 points, and performing closely to the much larger InternVL2-76B model.

InternVL 2.5 is an advanced multimodal large language model series based on InternVL 2.0. While maintaining the core model architecture, it introduces significant enhancements in training and testing strategies as well as data quality. This model explores the relationship between model scalability and performance, systematically investigating performance trends across visual encoders, language models, dataset sizes, and test settings. Comprehensive evaluations across a wide range of benchmarks, including interdisciplinary reasoning, document understanding, multi-image/video comprehension, real-world understanding, multimodal hallucination detection, visual localization, multilingual capabilities, and pure language processing, demonstrate InternVL 2.5's competitiveness comparable to leading commercial models like GPT-4o and Claude-3.5-Sonnet. Notably, it is the first open-source MLLM to achieve over 70% on the MMMU benchmark, attaining a 3.7 percentage point improvement through Chain of Thought (CoT) reasoning, showcasing strong potential for scalability during testing.

Llama-3.1-Nemotron-70B-Instruct is a large language model tailored by NVIDIA, focusing on improving the helpfulness of responses generated by large language models (LLM). This model excelled in various auto-alignment benchmark tests, such as Arena Hard, AlpacaEval 2 LC, and GPT-4-Turbo MT-Bench. It is trained using RLHF (specifically, the REINFORCE algorithm), Llama-3.1-Nemotron-70B-Reward, and HelpSteer2-Preference prompts on the Llama-3.1-70B-Instruct model. This model not only showcases NVIDIA's technological advances in enhancing generative support for general domain instructions but also offers a model conversion format compatible with the Hugging Face Transformers library, with free hosted inference available through NVIDIA's build platform.

Mistral-Nemo-Base-2407 is a 12B parameter large language model pre-trained by Mistral AI and NVIDIA. This model has been trained on multi-language and code data, and is significantly better than the existing models of the same or smaller scale. Its main features include: released under Apache 2.0 license, supporting pre-training and instruction version, 128k context window training, supporting multiple languages and code data, alternative product of Mistral 7B. The model architecture includes 40 layers, 5120-dimensional, 128 head-dimensional, 14364 hidden-dimensional, 32-head number, 8 kv-head (GQA), vocabulary size of about 128k, and rotative-embedding (θ=1M). The model performs well on multiple benchmarks, such as HellaSwag, Winogrande, OpenBookQA, etc.

Llama-3-70B-Tool-Use is a 70B parameter large language model optimized for advanced tool usage and feature call tasks. It achieves an overall accuracy of 90.76% on the Berkeley Feature Call Leaderboard (BFCL), outperforming all open-source 70B language models. This model enhances transformer architecture and is fine-tuned and trained with Direct Preference Optimization (DPO) on top of the Llama 3 70B base model. It takes text as input and produces text as output, with enhanced tool usage and feature call capabilities. While its main use case is tool usage and feature calls, it may be more appropriate for general knowledge or open-ended tasks to use a general language model. The model may produce inaccurate or biased content in some cases, and users should implement appropriate safety measures suitable for their specific use cases. The model is highly sensitive to temperature and top_p sampling configurations and requires proper adjustments to optimize performance.

OpenCompass 2.0 is a platform dedicated to evaluating the performance of large language models. It utilizes multiple closed-source datasets for multi-dimensional assessments, providing models with an overall average score and specialized skill scores. The platform helps developers and researchers understand the performance of different models in areas like language, knowledge, reasoning, mathematics, and programming through its real-time updated leaderboard.